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ABSTRACT 

A masters degree program for teachers of English as 
a Second Language is described, focusing on a method of student 
evaluation that emphasizes learner autonomy and seeks to extend that 
principle to trainees' classroom practice. The approach is based on 
the concepts that involvement in learning, including assessment, is 
vital to effective learning and that the teacher is the key person to 
help students devel op this learner autonomy. In the teacher training 
program described, the assessment scheme offers each course member a 
genuine opportunity to collaborate with their trainers in assessment 
of assignments. This is accomplished by using explicit criteria 
against which the trainee can evaluate his work, with negotiation of 
grade if the trainee's and trainer's assessments do not agree. This 
method balances the empowerment of the individual student to take a 
meaningful part in his own assessment with the need for the 
institution to retain responsibility for maintaining appropriate 
professional and academic standards. Comparison cf trainer and 
trainee assessment grades suggest that there is generally 
agreement , and interviews with students provide some evidence of a 
change in trainee attitude towards assessment and learner autonomy. 
(MSE) 
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COLLABORATION IN ASSESSMENT: 
EMPOWERING THE INDIVIDUAL 
COURSE MEMBER 

^ Leslie Dkkissoa 
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m 
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& Abstract 

The key to coping with change in education is learner autonomy, 
empowering the learner (in collaboration with the teacher) to adjust learning 
objectives, materials and techniques to suit changed circumstances. This paper 
describes an ongoing experiment at Moray House which seeks to give teachers and 
trainers on a post-experience Master's course experience of learner autonomy in the 
crucial realm of assessment, in the belief that this will help participants to develop a 
sympathetic attitude towards learner autonomy in their students. 

The students on the MA in TESOL at Moray House are invited to take part 
in a collaborative assessment scheme in which they have a genuine opportunity to 
contribute to the evaluation of their own work. At the same time the scheme 
ensures that the institution retains responsibility for maintaining appropriate 
standards and quality. 

Preliminary results suggest that there is broad agreement between the 
student being assessed and the tutor, and interviews with students provide some 
evidence of a change in attitude towards assessment and learner autonomy. 
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Introduction 



The argument in this paper is reasonably simple; it is as follows: 

- An important aim of language teacher education is to produce more 
effective language learning among pupils; 

- a key factor in learning effectiveness is having an active and independent 
involvement in learning; in other words, having a measure of learning 
autonomy. This is especially important in a fist changing world. 

- The key person in helping pupils to develop a measure of learning 
autonomy is the teacher. 

- However, developing autonomy is a fairly new idea and may be in 
conflict with the educational tradition. Consequently, some teachers and 
teacher trainees may not be in sympathy with the idea of autonomy in 
learning. 

- One factor in persuading teachers and trainee teachers to change their 
views is an experience of autonomy; this has the additional advantage of 
increasing their own learning autonomy. 

- This is done in the MA TESOL course at Moray House partly through an 
assessment scheme which offers each course member a genuine 
opportunity to collaborate with tutors in the assessment of his or her own 
assignments. 

- Data from monitoring the scheme confirm previous findings from 
research in self assessment, that there can be a reasonable level of accord 
between course member grades and tutor grades; and that the match 
between course member grades and tutor grades gets closer as course 
members gain experience of collaborative assessment. 



I will develop each of these arguments in the paper, though the main emphasis will 
be on describing the collaborative assessment scheme and discussing the data from 
it. 



THE IMPORTANCE OF DEVELOPING AUTONOMY 
^LANGUAGE LEARNING 

it. re is nowing evidence that success in languige learning, (ind in other subjects) 
dated to the leirner hiving in ictive, independent involvement with linguige 

f \nine I shill call this learning autonomy. Autonomous learners, are not 
«Jrilv or even characteristically isolated or independent learners; tbey may 
n be found in conventional classrooms, but they can be distinguished from 

S»er dependent learners in terms of several characteristics. Autonomous learners 

,re able to do the following things: 

- identify what is being taught That is, they are aware of the teacher's 

objectives; 

- state and follow-up their own purposes in addition to the teacher's. That 
is, they are able to formulate their own learning objectives; 

- select and implement appropriate learning strategies; 

- monitor their own learning; 

- monitor and evaluate their own use of learning strategies; 

The claim of the importance of learning autonomy can be supported both from 
research work in education, including language learning, and through reasoned 
argument. Wang and Peverly (1986) review findings of strategy research (in 
subjects other than language learning) and conclude 

"...one feature is salient across the research from the various perspectives. 
Effective learners are characterised in the research literature as being 
cognitively and affectively active in the learning process. They are seen as 
being capable of learning independently and deliberately through 
identification, formulation and restructuring of goals; use of strategy 
planning; development and execution of plans; and engagement of self- 
monitoring. (p.383) 

Similar findings have been suggested for language learning. The early research on 
language learning strategies carried out by such researchers as Rubin (1975), Stern 
(1975), and Naiman Frolich Stern and Todesco, (1978) indicated that good learners 
have an active involvement with language learning, that they have clear ideas about 
the best ways for them to go about language learning, and that they set up their own 



learning objectives in addition to the teacher's objectives. Groups like the Centre 
de Recherches et d 'Applications Pcdagogiques en Langues (CRAPEL) 1 at the 
University de Nancy II in France, and individuals like Ellis and Sinclair (1989) and 
Dickinson (1987) see language learning best facilitated by the development of 
greater independence on the part of the learner involving the learner in accepting a 
greater share of responsibility for his own learning. 

It is also possible to make an a priori argument in support of autonomy. Bruner 
(1966) states the argument very well 

Instruction is a temporary state that has as its -Hective to make the learner 
or problem solver self-sufficient..otherwise the ,esult of instruction is to 
create a form of mastery that is contingent upon the perpetual presence of 
the teacher. 

Bruner is saying that the outcome of instruction must, logically, be to make the 
learner self-sufficient or autonomous, since the alternative is the production of a 
learner who can only learn with the help of a teacher. 

Furthermore, modern society requires that the educated individual is able to adjust 
and change, and to continue learning in order to cope with the changing 
circumstances in a fast changing world, reflected in the theme of this seminar 
Thus, among the implications which Lange (1989) draws from his consideration of 
six very broad characteristics of the future "technological society"' are: 

3. Lifelong learning must be a construct in every teacher development 
programme. 

4. Experimentation, risk taking, autonomy, and flexibility must be key 
elements in the development of a model of schooling that places 
responsibility for learning on students, giving them freedom to try test 
innovate and create. 7 

Srr?"' 0 '! ROgCrS (1969) " gUCd * at lcami "8 10 ■» independent 
« a learner through learning how to learn is essential in a fast changing world. 

The only man who is educated is the man who has learned how to learn; 
the m,„ who has learned how to adapt ,„d change; the man who has 
realised that no knowledge is secure, that only the process of seeking 
Knowledge gives a basis for security. (104) 
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1 have argued that learning autonomy is an important outcome of education, is 
related to learning success, and that it is necessary in a fast changing world. I will 
now show that it is possible to give teachers and teacher trainees an experience of 
autonomy through involving them in self-assessment (which is in essence what 
collaborative assessment is about). 



WHAT IS COLLABORATIVE ASSESSMENT? 

The collaborative assessment scheme at Moray House has been used with various 
courses since 1985, and currently operates in the MA TESOL course for English 
language teacher trainers and language teachers. The central feature of the scheme 
is the possibility for course members to collaborate with tutors in the assessment of 
their own assignment work. 

The theory behind collaborative assessment is that motivating course members to 
make self assessments of their own assignments increases their autonomy. 
However, it is crucially important that this experience of self assessment is taken 
seriously by both the course member and the tutor. If it is seen merely as a 
simulation in which the course member's assessment of grade is ignored in the 
decision on the grade for the assignment, then course members are unlikely to take 
collaborative assessment seriously. The course member's grade has to be taken 
seriously, and the course member must feel that the grade he awards himself can 
make a difference in the decision on the final grade. On the other hand, the 
institution - represented in this instance by the course tutors - has the vital 
responsibility for maintaining the standards of the degree. If the course member's 
grade was accepted automatically then the grading system would quickly become 
meaningless, and the degree would rightly be seen as worthless. These two 
apparently conflicting positions can be resolved through the collaborative 
assessment scheme implemented in Moray House. 

In the scheme, each assignment is assessed using explicit criteria (provided by the 
tutors, but negotiated with the course members). At the point of handing in an 
assignment, the course member has the option of assessing her own assignment 
against the criteria and awarding herself a grade according to the marking scheme 
used in the course. This grade is recorded on the Collaborative Assessment Grade 
Sheet and submitted to the relevant tutor in a sealed envelope along with the 
assignment. The tutor then assesses the assignment and awards it a grade according 
to the criteria. 2 The tutor then, and only then, checks the Collaborative Assessment 
Grade. If the two grades are the same, then the agreed grade is entered in the 
record. If the two grades are different - either because the tutor gave a higher grade 
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than the course member, or because be gave a lower grade than the course member, 
then the tutor asks the course member to come and negotiate an agreed grade. 
However, the negotiation must be on the basis of the criteria. If the tutor and course 
member cannot agree, then there is provision for a third party to be involved, but 
that is rarely necessary. 

In this way, the scheme balances the empowerment of the individual course 
member to take a meaningful part in his or her own assessment, with the need for 
the institution to retain responsibility fcr maintaining standards appropriate to the 
level of masters degree. 

The scheme gives course members the opportunity to develop their autonomy 
through practising the evaluation of their own work and it gives them an experience 
of a measure of autonomy through this; involvement in self assessment. It raises 
awareness of several problems and issues in assessment which are important for 
teachers and teacher trainers to consider. Examples of such issues are the balance 
between formctive and summative functions of assessment; the power which 
assessment usually bestows upon the teacher/tutor; the relative merits of criterion 
referenced and norm referenced assessment, and the arbitrary nature of many 
decisions in assessment. In add ; /aon, it has the potential to increase the quality of 
the course members' work. This occurs through the process of making serious self 
assessments which requires more thorough familiarity with the assessment criteria 
because of the need to understand them thoroughly and examine them carefully in 
order to apply them in collaborative assessment If course members are involved in 
negotiation with the tutor over a grade, this also requires close familiarity with the 
criteria. 



HOW DOES COLLABORATIVE ASSESSMENT 
RELATE TO AUTONOMY? 

The essential characteristic of learning autonomy is that the learner makes (some of) 
the decisions about his or her own learning. That is, the learner retains a proportion 
of the responsibility for his own learning. So far as the MA TESOL course 
members are concerned, a crucial indicator of developing autonomy is the 
acceptance of responsibility for determining whether, and to what degree, a written 
assignment meets the standards of a Master's level course. The MA TESOL course 
attempts to provide a progression from the first Term, where course participants are 
given considerable counselling support, where criteria are carefully explained and 
where the first assignment - though graded for feedback purposes - is regarded as a 
practice assignment to help course participants to discover standards of the course, 



to the fourth Term, involving the writing and presentation of the dissertation (9 - 10 
monfrs after the beginning of the course). At this stage, the course participants are 
much more autonomous. 



Thus, the course participant's decision about whether his assignment work meUs 
appropriate standards develops over the year, up to and including the dissertation 
when most course participants are able and prepared to attest to the quality of their 
own work v« a vis the standards of the course This progression may continue into 
professional life after the conclusion of the courses where the emergent professional 
undertakes the presentation of papers at conferences and for publication; here the 
decision on the standard of the paper is (initially, anyway) for the writer to make. 
An essential difference between the expert and the novice lies in just this ability to 
make confident self-assessments. Experts can more confidently self-assess; novices 
ire still learning to do so. But novices do not turn into experts at the touch of a 
magician's wand; they develop overtime through the process of learning. 

The ability to assess oneself is a necessary outcome of any successful academic 
learning endeavour. It consists of 



Knowledge of the standards used by the academic community to judge 
performance at the appropriate level; 

the ability to compare one's own performance against those standards and 
place the performance on some scale. (e.g. That the performance is GOOD 
AVERAGE, POOR, NOT ACCEPTABLE.) 

The internalisation of the standards and the ability to judge future 
performance oneself against the internalised standards. 

I believe that all successful learning relates to self assessment in a similar way If 
one considers language learning, for example, a language element or unit can only 
be said to be learned when its performance by the learner is accepted by speakers of 
that language. However, the learner himself must achieve the ability to make some 
approximate self-assessment of the acceptability of his performance; otherwise he 
will be constantly uncertain of the acceptability of his performance and constantly 
teacher (or other informant) dependent 

Collaborative assessment of written assignments in the MA TESOL Course 
involves similar stages. It involves, learning about the appropriate level of 
Pertormancc, judged against some set of standards acceptable to the academic 
community. These standards are expressed - for a particular assignment topic -in 
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terms of the criteria supplied by the tutor, and learning to self-assess involves first 
understanding, sad then applying these criteria. A second stage is the establishment 
of generalised personal criteria based in part on the tutor's criteria, which can be 
used for any academic endeavour. It is perhaps at this stage, for some people, that 
the novice begins to turn into an expert 



WHAT EVIDENCE IS THERE THAT COLLABORATIVE ASSESSMENT 
WORKS? 

In order to be convinced that Collaborative Assessment works, one would want 
positive answers to a series of questions. Perhaps the key question to be answered is 
whether course members (OA'S) seriously attempt to assess themselves by applying 
the criteria in assessment, or do they merely try to bargain in order to get the best 
grade they can? This question can be answered in part by comparing the tutor's 
grade with the course member's grade. If there is a reasonable correlation between 
the grades awarded by tutors with those awarded by the course members, then this 
would be some evidence that course members are seriously attempting to assess 
themselves. 

The data in Table 1 shows the correlation between Tutor's grades and course 
members' grades for one group of MA students over the year. The correlation 
between the tutor's grades and the course members' for the first assignment is very 
low, (0.11; p = 0.57) indicating no relationship. However, the level of correlation 
over the next two assignments increases, though it does not reach statistical 
significance. 3 (For assignment 3 it was 0.33; p = 0.15; and for assignment 4 it was 
0.52; p = .009). 
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Notes; 



Labels: T:l = Tutor grade for Assignment 1; CA:1 = Course member grade for 
assignment 1. 

Sex; l=Male; 2=Female. 

Grades: Assignments are graded A-E. Numerical values have been assigned as 
follows A=5; B=4; C=3; D=2; E=l 

CORRELATIONS: 

Correlation of T:l and CA:1 = 0.1138, n=28, p=0.571 
Correlation of T:3 and CA:3 = 0.3253, n=21, p*0.147 
Correlation of T:4 and CA:4 = 0.5181, n=24, p=0.009 

One interpretation of the data * the one I prefer for obvious reasons - is that course 
members are seriously attempting to assess themselves against the criteria, and that 
they get better at it over the period of the course. We would expect this pattern; 
when course members begin the course, they have to discover the expectations of 
the tutors and the standards they are expected to achieve. The increasing 
correlations proceeding from the very low correlations of the first assignment to the 
much higher are for the third assignment may indicate that this learning process is 
going on. 

However, there are additional favourable indications that course members seriously 
attempt to assess themselves. We 4 collected the views of a group of students 5 on 
several aspects of assessment including the purposes of assessment and the 
authority of the tutor in assessment They were asked to complete a questionnaire 
right at the beginning of their courses, and before they had taken part in 
collaborative assessment 

We reasoned as follows: one view of the motivation of course members for taking 
port in Collaborative Assessment was that many do so in order to bargain for a 
better grade, without reference to what they thought they deserved. If it was the 
case that most students regarded the purpose of assessment as summative - that is 
more concerned with certification than with learning, then this would be evidence in 
support of the bargaining view. They would be less likely to make a sincere 
attempt to assess themselves against the criteria. We also reasoned that the course 
members' beliefs about the authority of tutors' decisions in assessment were crucial. 
If most students believed that tutors' interpretation of criteria, and their application 
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of them should be regarded as absolute, then it was less likely to be the case that 
participants in collaborative assessment were sincerely attempting to assess 
themselves. There would be little point since the tutors' decisions were irrefutable, 
and the only point would be to bargain for a better grade. 



Hie purpose of assessment 

The question of whether assessment is regarded by course members as a device for 
comparing students and testing their knowledge, or whether it is also perceived 
mainly as helping in learning was answered by all 34 coursi members. Just under 
half (47%) regarded "helping students to learn " as the most important reason for 
having assignments, and 29% placed it second in importance. None of those 
questioned regarded "comparing students" as an important reason for having 
assignments. 20.6% thought that the main reason for assignments was to test how 
much the students knew, and 26.5% regarded it as the second most important 
reason. Thus, there is no support here for the view that the main purpose of 
assessment is summative. The majority of these course members regarded the main 
purpose of assessment as formative, and therefore they are more likely to sincerely 
attempt to assess their own grade. 



The authority of the tutor in assessment 

We asked the course members about their view of the authority of the tutor at 
Moray House, and - since all the respondents to this questionnaire are teachers by 
profession - their view as teachers about their own authority over their pupils. We 
asked first about students in general; Did participants believe that students know 
what mark they deserve for a piece of work? 23.5% believed that students always 
know, 64.7% believed that students sometimes know, and 8.8% believed that 
students usually did not know what mark they deserved for a piece of work. We 
then asked about the respondents themselves; did tlie individual know what mark 
he or she deserved? As one would expect, the results were similar; 26.5% claimed 
to know always, 52.9% reported that they sometimes knew whilst 8.8% usually did 
not know. 

When we asked whether it was desirable for students to have this ability, 63.3% 
agreed that it was, while 30% had reservations and 6.7% believed that it was 
undesirable. 
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The participants were then asked whether tutors ought to help students to develop 
ability in self-assessment. 66.7% agreed that tutors should help students in this, 
whilst 21% thought that this was a possibility. 6.5% thought that it was a low 
priority (to be done only if there was time) and 6.5% believed that tutors should not 
help students to learn to self-assess. 

Taking these results together as an indicator of attitude towards the authority of the 
tutor in assessment, then about two thirds of the respondents thought that it was 
desirable for students to have the ability to estimate their marks, and a similar 
proportion saw it as an aspect of the tutor's job to help students to develop this 
ability. A high proportion of the respondents (88%) indicated a belief that students 
have opinions about how well they have performed in a particular assignment, and 
just under 80% reported that they themselves have reasonably reliable opinions 
about how well they have performed. 

If we can take these views as typical of course members taking the MA TESOL 
course (which is a big assumption), then taken as a whole, the results indicate that 
the majority of course members are likely to enter into collaborative assessment 
with the intention of genuinely attempting to estimate their own grades on the basis 
of the criteria, and that over the year they get better at doing so. 

Whatever the interpretation of this data, which anyway can only be illustrative, we 
believe that collaborative assessment is a valuable aspect of the MA TESOL course. 
It encourages course members to reassess their views about the purposes of 
assessment, and to re-consider their opinions on the respective roles of course 
members and tutors in assessment. My main theme was, of course, that 
collaborative assessment is an experience of autonomy, and that this experience of 
autonomy will help course members to develop a positive attitude towards learning 
autonomy in their own students, which is an important requirement for success in 
language learning, particularly in a fast changing world. 

NOTES. 

1. Sec, in particular Riley, P. (1965) 

2. We have a careful system of double marking lo ensure equitable standards among tutors. 
However, thia is not relevant in this paper. 

3. The correlations have been calculated for letter grades on a five point scale A • E; Tutor's grade 
and student's grade are regarded as the same only when there is an exact match e.g. both giving 
a 'C grade. One alternative which is sometimes used in Collaborative assessment (See Boud 
1987) is to accept grades as "the same" when they differ by up to 10%. We did not collect data 
which would allow ua to use the alternative. 
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4. The research being reported wis carried out by GUIies Haughton and myself. 

5. The data for collaborate* assessment of this group of students was not available when I wrote 
this; paper. 
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